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(57) Abstract: An auto-focusing method and device are presented for determining an in-focus position of a sample supported on 
a substrate plate made of a material transparent with respect to incident electromagnetic radiation. The method utilizes an optical 
system capable of directing incident electromagnetic radiation towards the sample and collecting reflections of the incident electro- 
magnetic radiation that are to be detected. A focal plane of an objective lens arrangement is located at a predetermined distance from 
a surface of the substrate, which is opposite to the sample-supporting surface of the substrate. A continuous displacement of the 
focal plane relative to the substrate along the optical axis of the objective lens arrangement is provided, while concurrently directing 
the incident radiation towards the sample through the objective lens arrangement to thereby focus the incident radiation to a location 
at the focal plane of the objective lens arrangement. Reflected components of the electromagnetic radiation collected through said 
objective lens arrangement are continuously detected. The detected reflected components are characterized by a first intensity peak 
corresponding to an in-focus position of said opposite surface of the substrate, and a second intensity peak spaced in time from the 
first intensity peak and corresponding to an in-focus position of said sample-supporting surface of the substrate. This technique 
enables imaging of the sample when in the in-focus position of the sample-supporting surface of the substrate. 
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AUTO-FOCUSING METHOD AND DEVICE 



FIELD OF THE INVENTION 

This invention is generally in the field of optical 
measurement/inspection techniques and relates to an auto-focus method and 
device, and an imaging system utilizing the same for use with optical 
5 microscopy. The invention is particularly useful for the microscope inspection 
of fluorescently labeled biological specimens on transparent slides, 

BACKGROUND OF THE INVENTION 

Auto focusing is an essential feature in many automated inspection 
fields such as the chip industry biomedical research, data reading/recording in 

10 optical information carriers, etc. 

Auto-focusing techniques used in the inspection/measurement of such 
structures as semiconductor wafers typically utilize an optical system separated 
from an imaging optics, which projects a light pattern (e.g., spots, lines, etc.) on 
the wafer and optimizes the image of this pattern (by using intensity contrast or 

15 displacement). Such techniques are applicable only for long working distance 
microscopes, i.e., with small numerical apertures. These techniques require 
calibrations and adjustments to match auto-focusing and imaging optical 
systems, and are hard to implement, especially when the imaging optics has 
variable magnification. U.S. Patent No. 5,925,874 describes a solution to the 

20 variable magnification problem based on projecting a focusing pattern through 
a magnifying imaging optics. 

Other common methods for auto-focusing in the chip industry 
inspection are based on the detection of tight reflected from a specimen, 
wherein incident light is focused either on the outer surface of the specimen, or 

25 on inner interfaces of the specimen (as in the case of infrared tight). 
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A recently developed auto-focusing technique for use with 
measurement/inspection on semiconductor wafers is disclosed in U.S. Patent 
No. 6,172,349. This technique is useful in both conventional microscopy and 
interferometry. According to Ihis technique, an operator designates the area 
within each field of view where the measurement has to be taken, and, for each 
area of interest (where the microscope is to be focused), translates the 
microscope along its optical axis (Z-axis) while measuring the image intensities 
at discrete sub-areas within the area of interest. These image intensities are then 
evaluated, and those having the greatest signal-to-noise ratio and occurring at a 
common point along the Z-axis are selected, and the corresponding sub-areas 
are identified. During subsequent inspections of the area of interest, only light 
reflected from the identified sub-areas is used to focus the microscope. 

Numerous publications (e.g., Born M. and Wolf, E. Principles of optics 
Cambridge University Press, 1997; T. Wilson and C. Sheppard, "Theory and 
practice of scanning optical microscopy", Academic Press, 1984) disclose 
optical principles, based on detecting the intensity variation above and below a 
focal plane (z-profile) for a point-like source, which are the basis for many 
auto-focusing systems. 

U.S. Patent No. 4,595,829 discloses a technique for use with reflected 
light microscopes. This technique utilizes the production of a measuring point 
on the surface of an object by an eccentric measuring beam formed by blocking 
a portion of the path of a full beam, and imaging the measuring point onto a 
photoelectric device by reflecting the measuring beam along the blocked-out 
path. 

U.S. Patent No. 4,844,617 discloses an auto-focusing technique for use 
in the inspection of semiconductor wafers. Here, a confocal microscope utilizes 
auto-focusing based on maximal intensity evaluated through an aperture larger 
than the diffraction limited size. 

U.S. Patent No. 5,288,987 describes a focusing technique for stereo 
microscopy, utilizing the imaging of an incident beam through a cylindrical lens 
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projecting a thin bar-shaped marking on a viewed object surface, in mis 
technique, imaging and focusing are not carried out via the same optics. 

U.S. Patent Nos. 5,783,814 and 5,672,861 describe focusing in a 
confocal microscope on the most reflecting layer by an iterative z-scanning of 
the intensity profile. 

U.S. Patent No. 6,128,129 discloses an auto-focusing technique for a 
microscope based on the optical path difference to a front focused position and 
to a back position that are conjugate with respect to the image forming point of 
the image forming optical system. 

U.S. Patent No. 4,798,948 describes an auto-focusing arrangement for 
use with incident light microscopes of dark field mumination. According to this 
technique, an incident light beam is comprised of several (at least two) partial 
beams which coincide in at least a central region of the incident light beam and 
which are of different wavelengths. An auto-focus measurement beam is 
presented by one of these partial beams. A lens element is inserted into the 
central region of the incident beam so as to change the beam cross-section 
(converging and diverging). 

Some auto-focusing techniques (e.g., US Patent Nos. 5,995,143 and 
5,790,710; and Geusebroek et al., Cytometry, 39:1-9 (2000) and references 
cited therein) are based on the analysis of image contrast and local intensity 
distributions. These techniques typically require the acquisition and analysis of 
many images to reach the best focus. The application of these techniques for 
fluorescence microscopy of biological samples is described in US Patent No. 
6,259,080, where the maximal signal to background value criterion is used. 
Here, a shutter is added to block the excitation mumination between image 
acquisition periods, thus minimizing the sample bleaching. The focusing 
technique of US Patent No. 5,790,710 is intended for use in a scanning 
microscope for inspecting fluorescently stained biological samples. This 
technique is aimed at avoiding bleach and phototoxicity utilizing the phase- 
contrast microscope image. 



WO 03/077008 



PCT/IL03/00206 



-4- 

Auto-focusing techniques utilizing an increase of the depth of focus of a 
focusing optics by digital combination of series of images taken at many close 
focal planes are disclosed, for example, in the following publications: Haeusler, 
G. and Koerner, E., "Imaging with Expanded Depth of Focus", Zeiss Inform., 
Oberkochen, 29:9-13, No.98E., (1986/87); Schormann, T. and Jovin, T.M., 
"Contrast Enhancement and depth perception in three-dimensional 
representation if differential interference contrast and confocal scanning laser 
microscope images.", J.Microscopy 166:155-168, (1992). According to these 
techniques, a plurality of images is taken and processed three-dimensionally to 
find the sharpest features of all focal planes which are presented in the single 
"deep focus" image. Additionally, these techniques are not preserving 
intensities, and therefore are not applicable for quantitative fluorescence 
microscopy applications. 

Present automated high-resolution (high-content) high-throughput 
applications of light microscopy in biology require scanning a large number of 
samples, such as cell yeast or bacteria, typically inside multi-well micro-plates, 
and acquiring images in each well, which resolve intracellular details at the 
utmost resolution possible by the optics. In order to resolve such details, setting 
the focus within the vertical focal resolution of the objective is mandatory. This 
means 0.3-0.5 (jm for immersed objectives, and 0.6-0.9 pm for high-numerical 
aperture long-working-distance air objectives. Such accuracy is far beyond the 
tolerance of the bottom surfaces of multi-well dishes, and even typical high- 
performance XYZ stages cannot scan distances of about 10 cm in the XY plane 
while keeping the Z coordinate within the above tolerances. 

A practical solution for the existing instruments is to find the focus for a 
given sample, based on the optimization of image sharpness. Different 
dgorithms, employing Fourier frequency space and real space image analysis, 
have been used, and Z scan, in a range that is expected to include the optimal 
focus, can guarantee that the best focal plane will be found (Geusebrock et aL, 
Cytometry 39:1-9 (2000) and references 1-8,10-11,25-26 cited therein). To 
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accelerate the process, the scan is first made in large steps, and refined till the 
required focus accuracy is reached. However, this approach has several 

inherent problems: 

1. The process is slow. Even if processing is performed in real time, at 
least an order of magnitude slower throughput is expected as compared to 
maximum possible image acquisition rate. 

2. A focus search algorithm could be based on efficient optimization. 
For example, Fibonacci minimization can reduce the range of the minimum 
search from 0.3mm to 0.3 pm in 12 steps. Yet, there is no scheme that will map 
the distance from the focus into any evaluation parameter with smooth 
dependence displaying a single minimum over three orders of magnitude range 
of defocusing. Scans must therefore start close to the focus (typically within 20 
jam) to refine its position 

3. Sharpness criteria are very sensitive to dirt, scratches and fingerprints 
on surfaces. Glass slides or plastic well bottoms are a fraction of a millimeter 
thick and may often cause auto-focus devices based on maximal sharpness to 
focus at the wrong side of the sample substrate. 

4. Sharpness algorithms are context-dependent, and there is no global 
focus criterion that can equally apply to a wide variety of image types. 
Fluorescently tagged cells may be found by the virtue of the cell-specific 
staining. However, image-based focusing is not applicable to fluorescent 
microscopy, where repeated imaging may cause bleach and phototoxicity. 
Fluorescence is localized also to specialized sub-volumes, thus the intensity 
varies greatly and is sample dependent. Typical biological screens based on 
fluorescent labels need to report intensities over a widely distributed range, as 
well as their absence. Focusing methods that evaluate fluorescence are 
therefore not applicable for such screens. 

5. Biological specimens (such as live cells) are basically transparent, and 
have very low reflectivity, usually from structures (such as organelles) 
dispersed through their whole volume. Transmitted tight microscopy can 
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acquire images by optical contrasting methods (such as DIC or phase contrast) 
that avoid photo-toxic fluorescence excitation. Yet, even if the focus reaches a 
plane somewhere within the extent of the cell height, typically 4-10 ,jm, some 
sub-cellular details may still be out-of-focus. To assure the covering of the 
whole cell height by three-dimensional (3D) optical sectioning methods, one 
needs to over-scan a focal range about twice as high as the cell, since the initial 
plane may be close to the bottom or the top of the cell. Phase images may have 
the best contrast at focal heights that depend on irregular structures within or 
above the sample, and hence, cellular details may still be blurred for high 
resolution imaging. Phase images may also produce the best contrast at under- 
or over-focus conditions due to the nature of phase-optics and due to the 
misahgnment of the phase ring optics. 

SUMMARY OF THE INVENTION 

There is accordingly a need in the art to facilitate the auto-focusing and 
inspection of samples, particularly biological samples, by providing a novel 
auto-focusing method and device, and an imaging system utilizing the same. 

The technique of the present invention provides for inspecting a sample 
supported on a plate-like substrate transparent with respect to incident 
radiation. The main idea of the present invention consists of detecting an in- 
focus position of the substrate's surface supporting the sample by continuously 
varying the relative distance between the sample and an objective (generally, 
focusing optics) along the optical axis of the objective so as to scan the 
substrate's thickness (height), while continuously directing incident radiation 
towards the substrate through the objective and detecting reflections of the 
incident radiation collected by said objective. This technique enables the 
sequential detection of two reflected intensity peaks, wherein the first detected 
peak corresponds to the in-focus position of the surface of the substrate closer 
to the objective, and the second detected peak corresponds to the in-focus 
position of the other opposite surface of the substrate, which carries the sample. 



WO 03/077008 



PCT7IL03/00206 



-7- 

This technique thus provides for a very fast in-focus positioning of the sample- 
carrying surface. 

There is thus provided according to one aspect of the present invention, 
an auto-focusing method for determining an in-focus position of a sample 
supported on one surface of a substrate plate made of a material transparent 
with respect to incident electromagnetic radiation, the method utilizing an 
optical system capable of directing incident electromagnetic radiation towards 
the sample and collecting reflections of the incident electromagnetic radiation 
which are to be detected, and comprising: 

(a) locating a focal plane of an objective lens arrangement at a 

predetermined distance from the other surface of the substrate, 
which is opposite to said surface supporting the sample; 

(b) providing continuous displacement of the focal plane relative to the 

substrate along the optical axis of the objective lens arrangement, 
while concurrently directing the incident radiation towards the 
sample through the objective lens arrangement to thereby focus the 
incident radiation to a location at the focal plane of the objective 
lens arrangement; and 

(c) continuously detecting reflected components of the electromagnetic 

radiation collected through said objective lens arrangement, said 
detected reflected components being characterized by a first 
intensity peak corresponding to an in-focus position of said opposite 
surface of the substrate, and a second intensity peak spaced in time 
from the first intensity peak and corresponding to an in-focus 
position of said sample-supporting surface of die substrate, thereby 
enabling imaging of the sample when in the in-focus position of the 
sample-supporting surface of the substrate. 
Thus, when reaching die in-focus position of the sample-supporting 
surface of the substrate, one or more images of the sample can be acquired with 
the substrate being stationary located with respect to the objective lens 
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arrangement. This technique is suitable for the case when cell-substrate 
contacts are of interest, and Ihey appear in focus at the level of the substrate 
interface. 

A series of images ("focal series" presentation of the 3D sample) can be 
acquired for different positions of the focal plane of the objective starting at the 
sample-supporting surface of the substrate using the in-focus position of the 
sample-supporting surface as a reference height. 

Alternatively, upon reaching the in-focus position of the sample- 
supporting surface of the substrate, an indication signal can be generated to 
actuate an imaging channel for a certain time period, while continuing the 
relative displacement between the focal plane of the objective lens arrangement 
and the substrate, through the sample. A series of pixel array images of 
successive planes of the sample along the optical axis of the objective lens 
arrangement is thus integrated on the imaging detector. This is the so-called 
"sweeping focus imaging mode", which results in data representation of the 2D 
projection of the 3D sample onto a two-dimensional pixel array of the detector. 

Preferably, the displacement of the focal plane relative to the substrate is 
carried out with a higher speed until the first intensity peak is detected, and is 
Ihen slowed until the second intensity peak is detected. Thus, a very fast 
approach to focus can be achieved from afar, but no mechanical overshoot will 
be experienced when reaching the in-focus position of the substrate's surface. 

When dealing with biological samples having fluorescent labels, the 
incident radiation used for auto-focusing is of a suitable wavelength range 
incapable of causing luminescent response of the sample. 

The sweeping focus technique of the present invention further provides 
for obtaining the real 2D-projection image of a 3D sample. This is implemented 
by two-dimensional deconvolution with the predetermined Point Spread 
Function of a sweeping-focus mode, thereby resulting in an image of the 2D 
projection of the 3D sample. 
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Thus, according to another aspect of the present invention, there is 
provided a method for obtaining an image of a three-dimensional sample in the 
form of a two-dimensional projection of the sample, which is supported on a 
substrate plate made of a material transparent for incident radiation, the method 

comprising the steps of: 

(a) providing data representative of images of successive planes of the 
sample along the optical axis of an objective lens arrangement 
continuously acquired by a two-dimensional pixel array of a detector 
during a continuous relative displacement between the focal plane of 
the objective lens arrangement and the substrate through the sample; 

(b) processing said data by carrying out a two-dimensional deconvolution 
thereof with the predetermined Point Spread Function of an imaging 
system, thereby obtaining said image in the form of the two- 
dimensional projection of the three-dimensional sample. 

According to yet another aspect of the present invention, there is 
provided an auto-focusing device for detennining an in-focus position of a 
sample supported on the surface of a substrate plate made of a material 
transparent with respect to incident electromagnetic radiation, the device 
comprising: 

- a light source generating a beam of the incident radiation of a 
predetermined wavelength range; 

- a focusing optics including an objective lens arrangement; 

a light directing assembly, which is operable for directing the incident 
beam towards the sample through the objective lens arrangement with 
a predetermined numerical aperture of beam propagation to irradiate a 
dif&action-bmited point on the focal plane of the objective lens 
arrangement, and for directing reflections of said incident radiation 
collected by said objective lens arrangement to a detector unit, which 
is operable to detect said reflections and generate data indicative of 
their intensities; 
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. a drive assembly operable to provide continuous displacement 
between the focal plane of said objective lens arrangement relative to 
the substrate along an optical axis of the objective lens arrangement; 
and 

- a control unit for operating said drive assembly to provide said 
continuous relative displacement and instantaneous stop of the 
displacement, for operating said light source and said detector to allow 
continuous detection of said reflections during said relative 
displacement, said control unit comprising a processing device 
operable to be responsive to said data generated by the detector unit to 
identify a first intensity peak corresponding to the in-focus position of 
a surface of said substrate opposite to the sample-supporting surface 
of the substrate, and identify a second intensity peak spaced in time 
from the first intensity peak corresponding to the in-focus position of 
the sample-supporting surface of the substrate, and to generate an 
auto-focusing signal upon detecting said second intensity peak. 
According to yet another aspect of the invention, there is provided an 
imaging system comprising said auto-focusing device and an imaging device, 
which comprises a light source and a detector. 

The present invention provides for a very fast auto-focusing technique. 
The focus swept time until the in-focus position of the sample-supporting 
surface is reached, depends practically on the time needed for mechanical focus 
adjustment, since the whole process is performed by closed-loop hardware 
control, e.g., a piezo-mounted objective having a millisecond response. The in- 
focus signal (intensity peak) is strong, falling sharply with me movement of the 
respective surface away from tiie in-focus position, and has a well defined 
magnitude which is independent on the sample and therefore can easily be 
identified by a very fast focal sweep, without being disturbed by weak signals 
associated with the sample. The magnitude of the signal is known (from a 
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suitable calibration procedure) and provides for identifying the in-focus 
position of the two opposite surfaces of the substrate. 

By using red (or infrared) light in die focusing apparatus, bleaching or 
photo-damage of the sample is avoided. The technique is therefore ideally 
5 suitable for inspecting live samples, such as GFP tagged cells. The method of 
the present invention is also suitable for thick samples, where no focal height of 
any absolute sense can be defined. The auto-focusing method of the invention 
sets on the substrate-slide surface forming the lower limit (closest to the 
objective) Z-focus of the 3D sample. The system offers a very high speed factor 
10 in the imaging and analysis of three-dimensional samples, since the exact edge 
of the cell volume (sample-supporting surface of the substrate) is determined 
avoiding the need for over-scanning focal planes. 

Preferably, the auto-focusing incident beam is expanded to fill the back 
aperture defined by the objective lens arrangement. The light directing 
15 assembly of the auto-focusing device comprises a beam splitting device for 
separating the paths of the incident and reflected light components propagating 
towards and away from the objective, respectively, through the auto-focusing 
channel. 

The imaging system preferably comprises a notch- or a high-pass 
20 wavelength-selective reflector, which separates the wavelength components 
propagating through the auto-focusing and imaging channels, in such a manner 
that the auto-focusing channel does not block a part of the imaging aperture. By 
using the identical optics (objective) for imaging and focusing and by filling the 
back aperture defined by the objective lens arrangement, the focus accuracy is 
25 automatically matched to the (m^ction-limited values characterizing the 
optimal imaging of the sample. 

The method of the present invention is applicable for transparent 
samples, such as biological cells. No reflection, scattering or fluorescence from 
the sample itself is used. The method is independent on the microscope imaging 
30 mode or sample type, and is effective even when some fields in the specimen 
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do not have cells or have cells that are not labeled. Being an imaging- 
independent method, 1he present invention can be used with various imaging 
modes (phase, Nomarskii Differential Interference Contrast (DIC) or 
fluorescence). The present invention can utilize air objectives, as well as the 
5 highest NA immersion objectives. The present invention as described below 
uses confocality principles, but it should be understood that it is operable with 
wide-field microscopes as well, without affecting the imaging optical path. 

For the inspection of a fluorescently labeled biological sample, an 
accessory red or infrared laser beam is employed to adjust the objective focus 
10 and find the substrate on which biological tissue cells are grown. The laser light 
can be switched on for focusing and switched off during imaging, or it may be 
left on permanently, and filtered out from the imaging optical path by the 
combination of the emission filter and the reflection in a specially designed 
wavelength-selective reflector which acts as a notch reflector for the laser 
15 wavelength, and is transparent to other colors. 

BRIEF DESCRIPTION OF THE DRAWINGS 

In order to understand the invention and to see how it may be carried out 
in practice, a preferred embodiment will now be described, by way of non- 
limiting example only, with reference to the accompanying drawings, in which: 
20 Fig. 1 schematically illustrates an imaging system for inspecting a 

biological sample that utilizes an auto-focusing device according to the 
invention; 

Figs. 2A to 2C illustrate the principles of the auto-focus detection 
method according to the invention, for three different examples, respectively, of 
25 air and immersion liquids filling the space between the objective and the 
substrate's surface supporting the sample; 

Fig. 3 is a flow diagram of the main operational steps in the auto- 
focusing method of the present invention; 
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Fig. 4 illustrates experimental results of the intensity profile of light 
reflected from different substrate-sample interfaces of the examples of Figs. 2A 
and 2B; 

Figs. 5A and 5B illustrate experimental results showing the role of a 
5 pinhole aperture located in the detector path of the auto-focusing device of Fig. 

i; 

Figs. 6A to 6C exemplify the results of simulating the invented 
technique using deconvolution for obtaining an image in me form of the two 
dimensional projection of a three-dimensional structure from the sweeping 
10 focus representation of the projection; 

Figs. 7A and 7B illustrate the theoretical results of calculating the 3D 
point spread function (PSF) of a microscope, and illustrate how the sweeping 
focus 2D PSF can be calculated from the 3D PSF; 

Figs. 8A to 8C compare the light intensity profile of the sweeping focus 
15 image of the present invention (shown in Fig. 7B) to that of the in-focus image 
of a point like source; 

Figs. 9A and 9B illustrate the main steps of the invented technique for 
obtaining the 2D-projection image of a 3D structure as compared to those of the 
conventional technique; 
20 Figs. 10A and 10B illustrate experimental results showing how the Z- 

information about the sample can be retrieved from two 2D-projection images 
of the sample simulating a stereo pair; and 

Figs. 11A to 11G illustrate examples of a cell stained for its 
mitochondria and a muscle cell stained for myosin, showing, for each cell 
25 respectively, a sweeping-focus 2D-deconvolved image of the samples, and a 
3D-deconvolved and projected image of the same samples; 



DETAILED DESCRIPTION OF THE INVENTION 

Referring to Fig. 1, there is illustrated an imaging system 10 for 
acquiring images of a sample, a biological sample BS in the present example. 
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The system comprises such main constructional parts as an imaging device 12 
and an auto-focusing device 14 associated with a control unit 15 connectable to 
the elements of me auto-focusing and imaging devices, as will be described 
more specifically further below. As shown, the biological sample BS (typically 
5 a sample chamber including a buffer layer in which cells are grown) is located 
on the surface S 2 of a substrate S (glass/plastic slide). The system 10 is located 
at the opposite surface Si of the substrate. The substrate is supported on a stage 
(not shown) movable in a plane perpendicular to me Z-axis. 

The control unit 15 comprises a processing utility PDi operable to 
10 process data coming from the auto-focusing device 14 to generate a focusing 
signal indicative of the in-focus position of the sample-carrying surface for 
actuating the imaging device 12. The latter can thus acquire images of the 
sample BS, when in the in-focus position thereof. The technique of the present 
invention also provides for obtaining an image of the two-dimensional 
15 projection of a three-dimensional sample. To this end, the control unit 15 
comprises a further processing utility PDi preprogrammed to carry out a two- 
dimensional deconvolution of detected data, as will be described further below. 
It should be understood that the processing utility PD 2 may be incorporated in a 
separate unit connectable to the output of an imaging detector. 
20 The imaging and auto-focusing devices 12 and 14 utilize a common 

objective lens arrangement 16 (which in the present example is composed of a 
single lens, but can generally be composed of more than one lens), and define 
imaging and auto-focusing channels of tight propagation. To facilitate 
illustration, the tight propagation through the imaging and auto-focusing 
25 channels is shown schematically: light component denoted Li and La propagate 
through the imaging channel, and tight component L3 and L4 propagate through 
the auto-focusing channel. As further shown in the figure, a shutter 19 is 
located in front of the light source 20, and is selectively operated to allow 
irradiation of the sample with the incident light Li to enable imaging of the 
30 sample. 
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The system 10 also comprises a suitable drive assembly operable by the 
control unit 15 to provide relative displacement between the objective 16 (i.e., 
its focal plane) and the sample BS along the optical axis of the objective (Z- 
axis) during the auto-focusing procedure (Z-scan). Generally, such a drive 
assembly can be associated with either one of the objective lens 16 and the 
sample BS. In the present example, the objective lens 16 is driven by a 
servomotor (e.g. piezo-drive) 17, operable by the control unit 15, for moving 
the objective along the Z-axis relative to the sample. 

Considering the specific example of the invention where the system 10 
is intended for inspecting the biological sample BS of the kind having 
fluorescent labeling, the imaging device 12 utilizes an appropriate light source 
20 (such as Mercury or Xenon arcs or Metal Halide lamps) for generating the 
incident light Li of a wavelength range capable of exciting the sample to cause 
a fluorescent response thereof la. The light response la is collected by the 
objective 16 and directed to an imaging detection unit 18 (e.g., comprising an 
area CCD associated with suitable imaging optics and imaging shutter 21). A 
light directing assembly of Ihe system 10 includes a wavelength-selective 
beam-splitting device 24 designed to reflect the wavelength range of the 
incident exciting light Li and transmit the wavelength range of the fluorescent 
excited light La, thereby preventing wavelength components of the incident 
exciting light Li from reaching the imaging detector 18 and allowing detection 
of the fluorescent light La- 

The substrate S is transparent with respect to incident radiation of both 
autofocusing and imaging wavelengths, as well as for the fluorescent excited 
light. The surface Si presents an interface between the glass/plastic substrate S 
and the environment E (e.g., air or immersion liquids). 

To spatially separate between light components propagating through the 
imaging and autofocusing channels, the light directing assembly of the system 
10 further employs a wavelength-selective beam spUtting device 22 (e.g., 
dichroic mirror, for example the Cat# 633DCRB STOCK #23416 model 
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commercially available from Chroma, VM). The beam splitting device 22 is 
constructed so as to transmit a wavelength range including the wavelength of 
the incident exciting light Li and that of the fluorescent response la, and to 
reflect any other wavelength range of light propagating through the auto- 

5 focusing channel (Tight components La and L4). 

The auto-focusing device 14 comprises a light source (laser) 26 
operating in an appropriate wavelength range - the red or infrared wavelength 
range in the present example (e.g., Uniphase 2mW He-Ne); a beam expander 
assembly 28; the beam directing assembly including a beam splitting device 38; 

10 the objective lens arrangement 16; and a detection unit 30. The use of red (or 
infrared) light for auto-focusing purposes avoids sample bleaching or photo- 
damage, and is therefore ideally suited for live samples, such as GFP tagged 
cells. 

The detection unit 30 comprises a detector 33 of the kind generating an 

15 electrical response to light signal impinging thereon (e.g., a PIN diode detector, 
for example the UDT PIN 10DF model commercially available from United 
Detector Technology), which is associated with a suitable imaging optics, 
composed of an imaging lens 34 and a confocal pinhole 36 in the present 
example. The provision of the pinhole 36 is aimed at rejecting (filtering out) 

20 more efficiently light components reflected from out-of-focus locations, and 
sharpening the detected intensity peak corresponding to the light components 
reflected from the in-focus locations. 

The beam splitting device 38 (typically a partially transparent mirror or 
50% reflecting cube) is used for spatially separating between the light beam L3 

25 emitted by the laser 26 and reflections thereof L,^ to Ihereby enable propagation 
of the reflections L 4 only to the detector 33. It should be noted that the use of 
the partially transparent mirror or 50% reflecting cube as the beam splitter 38 in 
the auto-focusing channel results in that 75% of the incident and reflected 
power is lost because of passing twice through the semi-transparent surface of 

30 the beam splitter 38. These losses can be reduced by using polarized light and a 
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polarization controlling assembly (polarizing beam splitter). More specifically, 
mis can be implemented by using a linearly polarized light Ls passing through a 
polarizing surface of the beam splitter 38, and a polarization rotating element 
(e.g., }J2 wave-plate) in liie optical palh of light U returned from the sample to 

5 adjust me polarization of light U to be reflected from the polarizing surface of 
the beam splitter 38 towards the detector unit 30. The linearly polarized light L 3 
can be achieved by using the highly polarized light source 26, or by using a 
polarizer in me optical path of randomly polarized light generated by the light 
source 26. Preferably, in order to use all the light generated by the light source, 

10 such a polarizer includes a polarization splitting means, a polarization rotator in 
a path of one polarization component, and a polarization combining means. The 
polarization splitting and combining means may be constituted by the same 
polarizing beam sphtter/combiner, provided an appropriate set of light reflectors 
is used. 

15 The expander assembly 28, which in the present example is composed 

of two lenses 31 and 32, provides for expanding the width of a beam Ls emitted 
by the laser 26 to thereby fill the back aperture defined by the objective lens 
arrangement 16. By changing slightly the distance between the two lenses in 
the assembly 28, it is possible to set the converging point of the laser beam Ls 

20 in front of or behind the imaging focus. The focal lengths of the two lenses 31 
and 32 determine their effect on the beam's cross-section width, and provide for 
matching the width of the beam so as to fill the entire back-aperture, and to 
enable the objective to focus the focus-sensing beam to a diflraction-limited 
size defined approximately by the equation: 

25 Res = "K/2NA 

wherein \ is the wavelength of the laser beam impinging onto the beam 
expander assembly, and NA = n(sinG) is the numerical aperture of the objective, 
n being the index of refraction of the environment E in front of the objective 
(e.g., air, or immersion liquids) and 9 being half the solid angle of the light 

30 beam after being focused by the objective. 
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Thus, the emitted beam of light L3 is expanded by the assembly 28 to be 
thereby capable of filling the back aperture defined by the objective lens 16. 
The expanded light beam L 3 propagates through the beam splitter 38 and is 
then reflected by the wavelength-selective surface of the device 22 towards the 
5 objective lens 16, which focuses the beam onto a diffraction-limited location on 
the focal plane of the lens 16. 

The auto-focusing procedure is aimed at locating the sample-supporting 
surface & of Ihe substrate in the focal plane of the objective lens 16. This is 
implemented by adjusting the focus of the objective to the first interface Si and 
10 thereby yielding a first approximation to the position of the second interface & 
at the other side of the substrate slide where cells grow. 

It should be understood that the imaging focus defined by the objective 
lens 16 (i.e., the laser auto-focusing point of convergence) and the confocal 
pinhole 36 in front of the auto-focusing detector are located in conjugated 
15 planes. By using the identical optics for imaging and focusing and by filling the 
entire numerical aperture of the imaging objective, the focus accuracy (z- 
resolution) is automatically matched to the diftraction-liniited values 
characterizing the optimal imaging of the specimen. On the other hand, the 
auto-focusing wavelength is split away from the imaging channel, and the 
20 detector 33 and pinhole 36 do not block any part of the imaging aperture. 

During the Z-scan (varying the distance between the objective and the 
substrate along the optical axis of the objective), the incident auto-focusing 
beam L 3 is sequentially reflected by substrate's surfaces Si and S2, and these 
sequential reflections L 4 propagate through the auto-focusing channel towards 
25 the detection unit 30 (i.e., pass through the objective lens 16 and are then 
reflected from the wavelength-selective surface of the device 22). The surface 
Si presents either one of air-glass, air-plastic and immersion liquid-glass 
interfaces, whatever the case may be, and the surface S 2 presents a glass-buffer 
or plastic-buffer interface. The fraction of light reflected at an interface is 
30 proportional to the difference in refractive indices of the two media at opposite 
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sides of the interface, which is about 4% for air-glass interface (i.e., surface Si 
which is located closer to the objective), and is about 0.8% for the glass-buffer 
interface (i.e., surface & on which cells are grown). For a 2mW HeNe laser 
used as the light source 26 in Hie auto-focusing device 14, the auto-focusing 
detector 33 behind the pinhole 36 should theoretically accept lO^W (calculated 
as the product of (2mW)x(4% refiection)xl/4) from the air-glass substrate 
reflection, and 4jj,W from the glass-buffer interface reflection. Experiments 
have shown that, with the 2mW energy of the emitted light beam L3, 0.7mW 
energy is reflected from the substrate's surface & and 7uW energy reaches the 
detector 33, which is accounted by the objective transmission (80%), slightly 
overfilling the back aperture, and selective reflector efficiency (80% twice). 

Reference is made to Figs. 2A-2C showing the principles of the auto- 
focusing technique of the present invention for three different examples of 
interfaces defined at the substrate's surfaces Si and S2. In the example of Fig. 
2A, a clean buffer sample BS (water or solution with n3=1.33) is located on the 
surface S2 of the glass substrate S (n 2 =1.52), and the environment at the other 
side of the substrate is air (ni=l). In other words, the surfaces Si and S2 present, 
respectively, air/glass and glass/water interfaces. In the example of Fig. 2B, the 
surfaces Si and S2 present air/glass and glass/cells-in-buffer (n 3 of about 1.33- 
1.35) interfaces. Fig. 2C presents an example wilh immersion/glass and 
glass/cells-in-buffer interfaces. 

Figs. 2A-2C show graphs G1-G3, respectively, representative of data 
continuously detected by the auto-focusing detector (33 in Fig. 1) during the z- 
scan. Each of these graphs is in the form of the intensity I(z) of the detected 
reflected light L4 as a function of the Z-positions of the focal plane of the 
objective 16 relative to the substrate S. These data are obtained while 
displacing the focal plane of the objective with respect to the substrate. At this 
stage, the imaging device is in its inoperative position, i.e., the shutter 19 in 
front of the imaging light source is closed. 
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As shown, each function I(z) is characterized by two intensity peaks Ii 
and I 2 with maximal values when the surfaces Si and S2, respectively, are at Zi 
and Z2 positions. These intensity peaks correspond to Ihe in focus positions of 
the surfaces Si and S 2 , since these surfaces are the only two boundaries of 
change in the index of refraction in the path of laser light L3 towards the 
specimen. It should be understood that the Z-scanning starts from the focal 
plane position a certain distance (of about one millimeter) from the substrate, 
and the peak Ii therefore appears first. The graphs G1-G3 differ from each other 
in the intensity values of the peaks, due to different interfaces, i.e., different 
values of (n 2 -ni) and (n3-n 2 ). 

The main steps in the auto-focusing method of the present invention will 
now be described with reference to Fig. 3. At the initiation of the auto-focusing 
session, the focal plane of the objective 16 is spaced from the substrate (its 
surface Si) a certain distance, e.g., is set to be of about one millimeter (step I). 
During the repeated automated setting of the focal plane positions along the 
substrate, this initial distance is set by the mechanical tolerance of the sample 
and stage height changes, which, for improved design, may be less than 100pm. 
The displacement of the focal plane of the objective is then handled by suitable 
hardware of the control unit 15, associated with the servomotor 17 displacing 
the objective as shown in the example of Fig. 1 (or associated with the 
substrate, whatever the case may be), and controlling the fflumination and 
camera exposure shutters 21. This hardware operates to provide a very fast 
movement of the focal plane of the objective towards the substrate (step H), 
until the intensity of the detected reflected light L 4 reaches its maximal value Ii 
(the first Z-derivative of the intensity is equal to zero) and the intensity value is 
higher then a preset threshold defined by the nature of the interface between the 
substrate and environment (step HI). This corresponds to the in-focus position 
of the substrate's surface Si. Then, the objective slows down (step TV) and sets 
a trigger (step VI) when the second peak I 2 is detected (step V), being defined 
by a respective threshold value in accordance with the environment This 
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second intensity peak corresponds to the in-focus position of the substrate's 
surface S 2 supporting the sample. It should be noted that it is possible to move 
the lens 32 with respect to lens 31 of the expander unit 28 and set the peak 
slightly before the imaging focus of the objective reaches the surface S 2 , thus 
enabling to set the initial focus at any prescribed distance before the actual cells 
are in focus, for example for over-height sampling purposes to cover out-of- 
focus contributions. The trigger may stop the focal displacement The control 
unit 15 thereby either actuates the imaging device 12 to carry out a 
computerized 3D acquisition of focal series, or continues the focal plane 
displacement through the sample, while the excitation and camera exposure 
shutters are opened to initiate a sweeping-focus image acquisition mode, as will 
be described below. The use of such an auto-focusing intensity pattern provides 
for accelerating the auto-focusing procedure, which is important for the high- 
throughput inspection of biological samples. 

Timing back to Figs. 2A-2C, a broader and noisy tail for the second 
peak h in the example of Fig. 2B associated to the side away from the 
objective is caused by the provision of a non-homogeneous medium of the 
buffer containing cells on the surface &. This does not affect the performance 
of detecting the peak when approaching from the objective direction. 
Notwithstanding these differences in the intensity pattern, it is evident that the 
method of the present invention provides robust focusing results for all three 
cases. 

Fig. 4 illustrates experimental results showing three intensity patterns 
H1-H3 (i.e., intensity changes during the z-scan) obtained by reflecting light 
from three glass slides (substrates), respectively, through a 40X microscope 
objective with a 0.75 numerical aperture and a 5pm pinhole. Graph Hi 
corresponds to a 1mm thick slide. In this example, the first peak only is 
detected, i.e., the focal plane position at the front surface closer to the objective 
(Si in Fig. 1). This is associated with the fact that the sweeping range is smaller 
than the slide thickness (the focal plane displacement is only over 0.5mm), the 
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focus thus never reached the sample-carrying surface of the substrate (&inFig. 
1). Graph Bb presents the case with a thin cover-slide (of a 180pm thickness) in 
air, i.e„ air-glass-air interfaces. Here, the first and second intensity peaks are 
shown (reflections from both surfaces Si and &) which are close in amplitude. 
5 Graph H3 is the intensity profile obtained with the same thin slide (160pm 
thickness) but with a water layer covering its back surface (& in Fig. 1), i.e., 
air-glass-water interfaces. As shown in graph Hb, the reflection from me glass- 
water interface (second peak) is 5 times weaker than that of the air-glass 
interface (first peak), which is in agreement with theoretical prediction. 
10 As indicated above, as the focus is swept, the auto-focusing detector (33 

in Fig. 1) records a sharp peak of intensity with a maximum when the first 
surface Si of the substrate (typically glass-air interface) is in focus. The width 
of the intensity peak, i.e., the fall rate of the intensity from the maximal value, 
is dependent on the defocus D (a distance between the plane defined by the 
15 substrate's surface Si and the focal plane of the objective, which is also the 
converging point of the laser light). The steepness of the peak can be regulated 
by the confocal pinhole aperture (36 in Fig. 1) in front of the detector. The role 
of the pinhole in the auto-focusing technique will now be more specifically 
described with reference to Figs. 5A and 5B. 
20 Fig. 5A illustrates the simulation of me intensity of the detected light as 

a function of defocus D. In mis example, the reflections L2 from the laser beam 
focused spot on the sample-supporting surface of the substrate are calculated. 
The calculations of the z-focus dependence of the Point Spread Function (PSF) 
intensity inside the circles were carried out using the Stokselh theoretical model 
25 (J. Opt. Soc. Am., 59, p. 13 14-1321, 1969). The intensity in the image was 
integrated in the appropriate pinhole size for the purpose of simulating variable 
pinhole measurement This configuration can actually be implemented in a 
digital microscope with minimal modification to find the substrate surface. To 
eliminate the detection of the laser reflections L 4 of the auto-focusing radiation, 
30 in case the emission filters are not sufficiently effective, the auto-focusing 
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channel must be turned off during imaging. In this case, however, the focusing 
process is longer due to the relatively slow reading rate of images. Three graphs 
(intensity peaks) R1-R3 are shown corresponding to different diameters of the 
annular pinhole apertures of, respectively, 0.5pm, 10pm and 5pm. It should be 
understood that these aperture dimensions are considered as the pinhole 
equivalents in me object plane. It is evident that the steepness of the decaying 
intensity peak can be broadened by opening the confocal pinhole aperture. 

Fig. 5B shows experimental measurements by the auto-focusing 
detector (33 in Fig. 1). Graphs Pi, Pz and P3 correspond to the intensity profile 
along the Z-axis for the pinhole openings of 50pm, 5pm and 0.5pm, 
respectively (i.e., pinhole equivalents in the object plane). For pinhole openings 
much larger than the diffraction-limited focused laser spot size, the detected 
intensity falls from its peak at the focus within a range comparable to the focal 
length of the objective. This range can be narrowed by closing the pinhole. 
Dynamic control of the pinhole opening facilitates the detection of the intensity 
gradient while approaching (fast) the in-focus location of the substrate plane 
from far away, and is used to tune the speed of this motion towards the in-focus 
position in a critically damped close loop. For the precise detection of the in- 
focus position, the pinhole should be closed to the diffraction size. 

As indicated above, when the in-focus position of the sample-supporting 
surface & of the substrate is detected, the control unit 15 actuates the imaging 
device 12. At this stage, one or more images of the sample at the in-focus 
position of the surface & and other positions at set distances from this surface 
can be carried out. To this end, the shutter 19 is shifted into its open position, 
and the light source 20 of the imaging device generates incident light Li to 
Muminate the sample and to allow the detection of the fluorescent response la 
by the detection unit 18. The output of the detection unit representative of data 
indicative of the detected light La is received and processed by the utility PB X 
of the control unit 15 to display one or more images of the sample. 
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The present invention also facilitates the imaging of samples (e.g., 
biological samples) by providing a solution for another problem associated with 
the following. The substrate surface is typically located at an unknown distance 
from other structures of interest inside the cell or on its cytoplasmic membrane 
due to cell height. In research studies with the conventional techniques, this 
problem is handled by three-dimensional optical sectioning microscopy, 
acquiring a series of focal images that can start at the substrate and continue to 
the sample maximal height. However, such a 3D data acquisition of many 
"optical sections" is slow, taking many seconds even in fast systems, yielding a 
huge volume of digital data, and the processing time puts heavy burden on the 
computing power. 

It appears that sufficiently good quantitative analyses, such as total 
fluorescence intensity evaluations and intracellular localization, can be obtained 
with the two dimensional projection of a three-dimensional structure. The 
reason for using three-dimensional image acquisition in the conventional 
systems (confocal or wide-field followed by deconvolution) is aimed at 
eliminating the complex out-of-focus contributions of high numerical aperture 
objectives. 

The sweeping-focus technique of the present invention provides an 
alternative fast way for obtaining an image of the 2D projection of a 3D 
sample. In order to obtain the 2D-projection image of the 3D sample, the 
system 10 of Fig. 1 operates in the following manner. The focal plane of the 
objective 16 is initially located far below the substrate' s surface Si, while the 
shutter 19 in front of the tight source 20 of the imaging channel is closed. The 
focal plane is continuously displaced towards the substrate. When the second 
intensity peak is detected (i.e., the sample-supporting surface & of the substrate 
is in focus), the shutters 19 and 21 are opened, and the displacement of the 
focal plane along the Z-axis is continued, thereby enabling acquiring images of 
the sample integrating data over a range of heights within the specimen volume. 
Preferably, the shutter 19 should be opened somewhat prior to the detection of 
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the second peak, to enable recording out of focus contributions from regions 
outside the specimen volume. The time at which the shutter must be open (At) 
depends on the speed of the focal plane displacement (v f ) and the expected 
specimen height (hs): At = Wvf. The shutter is opened during the Z-scan of the 
sample's height and, preferably, a bit longer time to avoid losing information in 
both edges of the sample. The detection unit 18 integrates light coming from 
the sample during the continuous movement of the focal plane through the 
sample height along the optical axis of the objective. The so-measured image 
data is thus representative of the 2D projection of the 3D sample, but includes 
also the out-of-focus contributions. This data representation is received at the 
processing utility PD2 where it undergoes two-dimensional deconvolution, 
resulting in the 2D-projection image of the sample. The results of this technique 
are comparable to the conventional technique (i.e., 3D deconvolution of a 3D 
representation followed by a 2D projection), and are obtainable wilh much 
higher image acquisition and post acquisition processing speeds. 

The focal depth is inversely proportional to the numerical aperture NA 
of the focusing/collecting optics. In the extreme case, NA-0 (i.e., the focal 
depth is infinite), the image is a parallel beam projection, such as for Electron 
Microscopy. However, sub-cellular resolution light microscopy is inevitably 
associated with high numerical aperture imaging with very limited focal depth, 
as well as complex behavior of out-of-focus contributions. The contributions of 
out-of-focus patterns in an imaged sample cause loss of contrast (and effective 
resolution). Although confocal microscopes practically provide for reducing 
these contributions, the quantitative evaluation of the location and amount of 
labeled molecules in a biological sample is impeded by the intensity variations 
with distance from the focal plane, unless many, very closely spaced focal 
planes are acquired. This is not only slow and heavy on the computer, but also 
causes excessive bleach. Therefore, quantitative microscopy typically utilizes 
wide-field image acquisition where integrated intensities vary little from the in- 
focus image (just distributions are blurred). The sweeping focus acquisition 
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maintains this feature, preserving the contributions of fluorescent labels 
throughout the sample volume. 

Computational projection and 2D deconvolution of a focal series data 
was used for time-lapse data analysis (Hiraoka et al., Focal points for 

5 chromosome condensation and decondensation revealed by three-dimensional 
in vivo time-lapse microscopy, 1989, Nature 342, 293-296). With this 
technique, final results similar to those of 3D deconvolution followed by 2D 
projection technique were achieved, but at much greater speed and reduced 
load on computer storage and processing time. Here, the present invention also 

10 provides for dramatically cutting the image acquisition time, and enables high- 
throughput high-resolution microscopic screening of samples, such as light- 
microscopy for sub-cellular localization and quantification of fluorescently 
labeled molecules, notably, cells expressing inherently fluorescent proteins 
chimeric constructs. 

15 The following is the description of the 2D-projection imaging technique 

utilized in the present invention. In this connection, it should be understood that 
convolution and projection are commutative linear operations, meaning that 
their order of activation can be exchanged without affecting the final result. A 
relation between a 3D microscope object 0 3 (x,y,z) and an acquired 3D image 

20 thereof M 3 (x,y,z) is expressed by a 3D convolution as follows: 

M 3 (x,y,z) = ffl dx'dy'dz' 0 3 (x\y',z') * P 3 (x-x',y-y\z-z')= (1) 
= 0 3 (x,y,z)® 3 P 3 (x,y,z) 

25 

wherein P 3 (x,y,z) is the convoluting kernel defined by the microscope 3D Point 
Spread Function (PSF), which is assumed to be independent of position (i.e., 
the so-called space variant); and <g> 3 denotes a 3D convolution operation. 

A 3D PSF is actually the 3D image of a point source resulted from 
30 imaging it at successive focal planes along the optical axis of the objective from 
to +oo, (practically for 100X objective, from 5pm above the focal point to 
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5,jm below it). The z-projection of a 3D image is a 2D image given by the 
integral over z, and similar for the PSF and the object: 

M 2 (x,y) = J dz M 3 (x,y,z) (2a) 

P 2 (x,y)=JdzP 3 (x,yz) (2b) 
0 2 (x,y) = ldz0 3 (x,y,z) ( 2c > 

Integration of the equation (1) along the z-axis gives: 
MW) = 0 2 (x,y) ® 2 PW) ( 3 > 



Equation (3) signifies that a measured sweeping focus image, M (x,y), is 
related to the projection of the real 3D object, 0 2 (x,y), by a 2D convolution, 

15 and the convolution kernel P^y) is the z-projection of the microscope 3D 
PSF. It is therefore possible to invert this relation and use P 2 (x,y) in a 2D 
deconvolution that retrieves the projection from the sweeping focus measured 
presentation. Such a processing is commonly executed by iterative constraint 
deconvolution algorithms. 

20 Reference is made to Figs. 6A-6C showing the simulated results of 

obtaining the 2D-projection image of a simulated 3D structure composed of 
points, lines and triangular planes at random positions, heights and orientations. 
Fig. 6A illustrates a computed 2D-projection (z-projection) of the original 3D 
simulated image of this structure. Fig. 6B illustrates simulation of the 

25 sweeping-focus imaging of this structure, and presents a 2D-projection 
representation of the structure. This data representation was obtained by first 
convolving the 3D image with the microscope 3D PSF, and then projecting the 
convolution result along the Z-axis. As shown, the 2D data representation is in 
the form of a blurred image caused by the effects of the out-of-focus patterns of 

30 the structure. Fig. 6C illustrates the results of processing the 2D-projection 
representation of Fig. 6B by a 2D deconvolution. As shown, the blur is 
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significantly reduced The results of this technique (2D projection of Fig. 6C) 
are comparable to Fig. 6A, presenting the result achieved by the conventional 
3D-image acquisition followed by a 3D deconvolution and projection along the 
Z-axis, other than the effect of the smear due to the resolution limit in-focus 
conditions. The 2D deconvoluted image actually presents the infinite depth-of- 
focus picture of a thick sample. The depth information within the sample is lost, 
but the fluorescence intensity is proportional to the number of fluorophores in 
Ihe entire volume, and the sharpness of an object within the specimen volume 
approaches the lateral (X-Y) resolution defined by the in-focus conditions, 
independent of its Z-position. The information along the Z-axis can be retrieved 
by acquiring two sweeping focus images during translation of the sample stage 
in two opposite directions (±X), thereby creating a stereo-pair, as will be 
described further below. 

Reference is now made to Figs. 7A and 7B showing how the 2D- 
sweeping focus PSF can be obtained experimentally and by calculations from 
theoretical models. Experimentally, sweeping focus images of sub-resolution 
sized fluorescent beads are simply acquired exactly as the sweeping-focus 
images are. Theoretically, any model for the microscope 3D PSF can be 
projected along the Z-axis. Fig. 7A shows a 3D image of a point source, i.e., 
PSF of die microscope, calculated according to the model of Stokseth (J. Opt 
Soc. Am., 59, p. 13 14-1321, 1969). The 3D image includes seven optical 
sections, a central one corresponding to the in-focus position of the point 
source, So, three optical sections above the focus, and three optical sections 
below the focus. At the in-focus position of the point source (plane Z=0), the 
image is in the form of a diffraction-limited point So. When the focal plane of 
the objective is displaced certain distances in two opposite Z-directions from 
the in-focus position of the source point, the image is blurred, displaying 
pronounced concentric ring structures (Airy rings). Fig. 7B illustrates the 
intensity profile of the 2D projection of the 3D image of Fig. 7A (a line scan 
profile through the image center in Fig. 7B), showing how the intensity falls 
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from its maximum at the center. The central point has roughly the same 
diffraction-limited diameter as the in-focus image, and is surrounded by 
decaying lobes, that totally averaged the ring structure (since it is continuously 
expanding as a function of distance from focus). 

5 Figs. 8A to 8C compare the light intensity profile of the sweeping focus 

image of a point source obtained with the sweeping focus method of the present 
invention (graphs Bi) with that of the in-focus image of a point source (graphs 
B2). Fig. 8A exemplifies the normalized line scan profile at the image center; 
Fig. 8B exemplifies the four-times magnified profile visualizing the out-of- 

10 focus contribution; and Fig. 8C shows the logarithmic scale of the same (three 
decimal decades). 

Referring to Figs. 9A and 9B, there are illustrated the main operational 
steps of a method according to the invention for obtaining the 2D projection of 
a 3D object (Fig. 9A) as compared to that of the conventional technique (Fig. 

15 9B). According to the invention (Fig. 9A), a sweeping focus image acquisition 
is applied to a 3D object (step A), thereby obtaining a 2D-projection 
representation (step B), which then undergoes a 2D deconvolution using 
sweeping focus PSF (step C) resulting in the 2D projection image. According to 
the conventional technique (Fig. 9B), a series of focal images of the 3D object 

20 are acquired resulting in a measured data in the form of a series of optical 
sections across the object Then, this measured data undergoes 3D 
deconvolution using a 3D-PSF, thereby reconstructing the 3D object and 
enabling the calculation of a 2D projection thereof. 

It should be understood that, for the purposes of the present invention, 

25 the 2D projection of the 3D-PSF, P 2 (x,y) 5 should be measured or theoretically 
calculated (according to equation (2b) above) only once, for a given apparatus 
at a given configuration. Since data representative of a typical 2D image of a 
biological sample occupies about 2 Mbytes (in comparison to 100 Mbytes of 
that of a 3D image), the time saving in calculation is enormous. The trade off is 
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loosing all Hie information about the z-component of me data, which can Ihen 
be retrieved in the following manner: 

Figs. 10A and 10B show two 2D-projection images L and R, 
respectively, obtained by acquiring sweeping focus images of beads at different 
heights while translating the microscope stage (substrate S in Fig. 1) in two 
opposite (+x) directions, thereby creating a stereo-pair. The ratio between the 
Z- and X-speeds of translation is defined as F. The two images are then 
processed by deconvolution. Strictly speaking, such sweeping of focus together 
with shifting X-position has its special 2D PSF functions (easily calculated 
from the 3D PSF by the projection at an axis tilted with respect to the Z-axis). 
The results, however, are sufficiently close to the unfitted projection, which can 
be used as an excellent approximation. An algorithm of reconstructing the Z- 
position of continuous objects from stereo pairs of electron micrographs has 
been developed (Cheng, Y, Hartemink, C.A., Hartwig, J.H. Forbes Dewey Jr. 
C, Three-dimensional reconstruction of the actin cytoskeleton from stereo 
images. J. Biomechanics, 33, p. 105-113, 2000). Defining the Z-position of 
highly localized labeled spots is particularly simple, since a relative shift in 
position from image L to image R is proportional to the height In the image L, 
points A, B and C are shown, which signify the following: point A is the 
reference bead (i.e., Z=0 corresponding to the in-focus position of the bead's 
plane); point B is a bead that shifts (-xi) between the images L and R, therefore 
being at "height" of (-xi-F)=(-Zi); P oint c presents a smaller (+x 2 ) shift 
between the images L and R, the "height" of the point C therefore being 

(X2-F)=Z2. 

The post-acquisition, computational method of obtaining the 2D- 
projection image of a 3D object by 3D acquisition, projection, and then 2D 
deconvolution (e.g., from the above indicated article of Hiraoka et al), is 
implemented in the present invention by hardware, using the sweeping focus 
method. This serves for saving the time needed for acquisition and processing 
of a large number of images (e.g., 64 section images). In the invention, based 
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on equation (3) above, the 2D deconvolution (step C in Fig. 9A) is preferably 
based on iterative constrained deconvolution (Agard D.A. and Sedat, J.W. 
(1983) Nature 302, 676-681; Agard D.A, Optical sectioning microscopy: 
cellular architecture in three dimensions, Ann Rev. Biophys. Bioeng., 13, 
p. 191-219, 1984; Swedlow J.R, Sedat J.W., Agard D.A in: Jansson, P.A 
editor, Deconvolution of images and spectra, Academic Press NY, chapter 9, 
pp.284-309, 1997), which is conveniently computed in the Fourier space, and 
avoids dividing by small values. The convolution of two 2D functions 
(indicative of the object and the sweeping focus 2D PSF) has algorithmic 
complexity of the order of N 4 , N being the number of pixels in one dimension, 
while in the Fourier space, the procedure (using Fast Fourier Transform) is of 
complexity of N^QS 2 ~ N 2 (lnN) only. Therefore, the Fourier transforms of 
equation (3) can be carried out, wherein the convolution in the real space is 
replaced by multiplication in Fourier space. At each iteration, a guess image is 
convoluted with the PSF, compared to the measured sweeping focus 
presentation, and Ihe deviations are used to correct the guess, assure its 
positivity, and continue to next iteration. 

The following are examples of experimental results of the technique of 
the present invention. 3D focal series and sweeping focus 2D projection images 
were acquired from fluorescentiy labeled specimens. 3D constrained 
deconvolutions were calculated (Agard D.A., Optical sectioning microscopy: 
cellular architecture in three dimensions, Ann. Rev. Biophys. Bioeng., 13, 
p. 191-219, 1984; Swedlow J.R, Sedat J.W., Agard DA. in: Jansson, PA 
editor, Deconvolution of images and spectra, Academic Press NY, chapter 9, 
pp.284-309, 1997). Then, deconvoluted 3D result was processed by computed 
projection, and compared to the sweeping focus acquired image of the same 
samples followed by 2D deconvolution with the sweeping focus PSF. This is 
illustrated in Figs. 11 A to 11G. Figs. 11 A and 11B illustrate sweeping focus 
images before and after deconvolution, respectively, for a mitochondria. Fig. 
11C is the 3D deconvoluted and projected corresponding image. Figs. 11D and 
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11E illustrate sweeping focus images before and after deconvolution, 
respectively, for a muscle cell sample. Fig. 11F is the 3D deconvolved and 
projected respective image, and Fig. 11G is me enlarged view of the image of 
Fig. 11F. 

Those skilled in the art will readily appreciate that various modifications 
and changes can be applied to the embodiment of the invention as hereinbefore 
exemplified without departing from its scope defined in and by the appended 
claims. It should be understood that, although the present invention is described 
herein with respect to a specific example of imaging biological samples, the 
technique of me present invention can be used for inspecting samples of other 
types, provided a sample under inspection is located in a plate-like substrate 
transparent with respect to predetermined incident radiation. Considering the 
entire imaging system (10 in Fig. 1), it should be understood that for samples 
other than those having fluorescent labels, imaging and auto-focusing channels 
could utilize the same light source, and therefore a light directing assembly 
would be simpler. 
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CLAIMS: 

1. An auto-focusing method for determining an in-focus position of a 
sample supported on one surface of a substrate plate made of a material 
transparent with respect to incident electromagnetic radiation, the method 
utilizing an optical system capable of directing incident electromagnetic 
radiation towards the sample and collecting reflections of the incident 
electromagnetic radiation which are to be detected, and comprising: 

(i) locating a focal plane of an objective lens arrangement at a 
predetermined distance from a surface of the substrate, which 
is opposite to said surface supporting the sample; 

(ii) providing continuous displacement of the focal plane relative to 
the substrate along the optical axis of the objective lens 
arrangement, while concurrently directing the incident 
radiation towards the sample through the objective lens 
arrangement to thereby focus the incident radiation to a 
location at the focal plane of the objective lens arrangement; 
and 

. (iii) continuously detecting reflected components of the 
electromagnetic radiation collected through said objective lens 
arrangement, said detected reflected components being 
characterized by a first intensity peak corresponding to an in- 
focus position of said opposite surface of the substrate, and a 
second intensity peak spaced in time from the first intensity 
peak and corresponding to an in-focus position of said sample- 
supporting surface of the substrate, thereby enabling imaging 
of the sample when in the in-focus position of the sample- 
supporting surface of the substrate. 




# 
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2. The method according to Claim 1, wherein said incident radiation is in 
a spectral range incapable of causing luminescent response of the sample 
having luminescent labels therein. 

3. The method according to Claim 1 or 2, wherein said displacement is 
5 started from a certain distance between the focal plane and said opposite 

surface of the substrate. 

4. The method according to any one of preceding Claims, wherein the 
displacement of the focal plane is slowed upon detecting said first intensity 
peak. 

10 5. The method according to any one of the preceding Claims, and also 

comprising providing a relative displacement between the optical axis of the 
objective lens arrangement and the substrate in a plane perpendicular to said 
optical axis, to thereby provide intersection of the optical axis with successive 
locations of the substrate in said plane, and repeating steps (ii) and (iii) with 

15 respect to the successive locations of the substrate. 

6. The method according to any one of preceding Claims, wherein a 
numerical aperture of the incident beam propagation towards the objective lens 
arrangement is such as to fill a back aperture defined by the objective lens 
arrangement. 

20 7. The method according to any one of preceding Claims, comprising 

spatially separating the incident and reflected radiation components, to allow 
detection of the reflected radiation. 

8. The method according to any one of preceding Claims, comprising 
passing the collected reflected radiation, propagating towards a detector, 

25 through a pinhole, thereby filtering out radiation components reflected from out 
of focus locations and sharpening the detected intensity peak corresponding to 
the light components reflected from the in-focus locations. 

9. The method according to any one of the preceding Claims, comprising 
imaging the sample when in the in-focus position of the sample-supporting 

30 surface of the substrate. 
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10. The method according to Claims 2 and 9, wherein said imaging 
comprises irradiating me sample having luminescent labels by incident 
electromagnetic radiation of a wavelength range capable of exciting a 
luminescent response of the sample, detecting said luminescent response by an 
imaging detector, and generating data indicative thereof. 

11. The method according to Claim 10, wherein said incident exciting 
radiation is directed to the sample and the luminescent response is directed to 
the imaging detector, through said objective lens arrangement. 

12. The method according to Claim 11, further comprising spatially 
separating the luminescent response and the reflected components to allow 
detection of the luminescent response of the sample by said imaging detector 
and prevent the reflected components from reaching the imaging detector. 

13. The method according to any one of preceding Claims, further 
comprising: 

- performing a sweeping-focus acquisition mode continuously 
acquiring images of successive planes of the three-dimensional 
sample along the optical axis of the objective lens arrangement, by 
continuously displacing the focal plane of the objective lens 
arrangement through the sample, starting at the sample-supporting 
surface of the substrate, thereby obtaining data representation of the 
two-dimensional projection of the three-dimensional sample onto a 
two-dimensional pixel array of a detector; 

- processing said data representation by carrying out a two-dimensional 
deconvolution thereof with a predetermined Point Spread Function of 
a sweeping-focusing acquisition mode, thereby obtaining a deblurred 
image of the two-dimensional projection of the three-dimensional 
sample. 

14. The method according to Claims 2 and 13, wherein the sweeping 
focus acquisition mode comprises continuously irradiating the sample having 
luminescent labels therein with incident exciting radiation causing luminescent 
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response of the sample, and continuously detecting the luminescent responses 
coming from the successive planes of the sample. 

15. An auto-focusing device for determining an in-focus position of a 
sample supported on the surface of a substrate plate made of a material 
transparent with respect to incident electromagnetic radiation, the device 
comprising: 

a light source generating a beam of the incident radiation of a 
predetermined wavelength range; 

- a focusing optics including an objective lens arrangement; 

- a light directing assembly, which is operable for directing the incident 
beam towards the sample through the objective lens arrangement with 
a predetermined numerical aperture of beam propagation to irradiate a 
location on the focal plane of the objective lens arrangement, and for 
directing reflections of said incident radiation collected by said 
objective lens arrangement to a detector unit, which is operable to 
detect said reflections and generate data indicative of their intensities; 
a drive assembly operable to provide continuous relative displacement 
between the focal plane of said objective lens arrangement and the 
substrate along the optical axis of the objective lens arrangement; and 
a control unit for operating said drive assembly to provide said 
continuous relative displacement and instantaneous stop of the 
displacement, for operating said light source and said detector to allow 
continuous detection of said reflections during the displacement of the 
focal plane, said control unit comprising a processing device operable 
to be responsive to said data generated by the detector unit to identify 
a first intensity peak corresponding to the in-focus position of the 
surface of said substrate opposite to the sample-supporting surface of 
the substrate, and identify a second intensity peak spaced in time 
from the first intensity peak and corresponding to the in-focus position 
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of the sample-supporting surface of the substrate, and to generate an 
auto-focusing signal upon detecting said second intensity peak. 

16. The device according to Claim 15, wherein said predetermined 
wavelength range of the incident radiation is incapable of causing luminescent 

5 response of the sample having luminescent labels therein 

17. The device according to Claim 15 or 16, wherein said light directing 
assembly comprises a beam splitter for spatially separating between the 
incident and reflected radiation components, and allowing propagation of the 
reflected radiation components towards the detection unit 

10 18. The device according to any one of Claims 15 to 17, wherein the light 

directing assembly comprises a beam expander for providing said 
predetermined numerical apertures of the incident beam propagation to fill a 
back aperture defined by the objective lens arrangement 

19. The device according to any one of Claims 15 to 18, wherein said 
15 detection unit comprises a pinhole accommodated in the path of the collected 

reflected radiation propagating to a detector. 

20. The device according to any one of Claims 15 to 19, wherein said 
control unit is operable to slower the displacement of the focal plane, upon 
detecting said first intensity peak. 

20 21. The device according to any one of Claims 15 to 20, wherein said 
control unit is operable to actuate an image acquisition mode to acquire images 
of the sample when in the in-focus position of the sample-supporting surface of 
the substrate. 

22. The device according to Claim 21, wherein the control unit is 
25 connectable to an imaging device operable to carry out said image acquisition 

mode. 

23. The device according to any one of Claims 15 to 20, wherein said 
control unit is operable to actuate the drive assembly to provide a continuous 
displacement of the focal plane relative to the substrate through successive 

30 planes of the substrate along the optical axis of the objective lens arrangement 
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starting from the sample-carrying surface, and to actuate an image acquisition 
mode to continuously acquire images of the successive planes of the sample. 

24. The device according to Claim 23, comprising a processing device for 
processing data indicative of said images to obtain a two-dimensional 
projection of the three-dimensional sample. 

25. An imaging system comprising the auto-focusing device of Claim 15, 
and an imaging device utilizing said objective lens arrangement and denning an 
imaging channel for electromagnetic radiation propagation towards and away 
from the objective lens arrangement separated from the auto-focusing channel 
defined by the auto-focusing device, said imaging device comprising a light 
source generating incident radiation and an imaging detection unit. 

26. The system according to Claim 25, wherein said predetermined 
wavelength range of the incident radiation used in the auto-focusing device is 
incapable of causing luminescent response of the sample having luminescent 
labels therein, said imaging device comprising a light source generating 
incident exciting radiation to excite a luminescent response of the sample, said 
response being collected by the objective lens arrangement and directed to the 
imaging detection unit. 

27. The system according to Claim 26, comprising a wavelength selective 
beam splitter accommodated in the paths of the collected reflected radiation and 
the collected luminescent response to provide said spatial separation between 
the radiation components propagating through the imaging and auto-focusing 
channels. 

28. A method for obtaining an image of a three-dimensional sample in the 
form of a two-dimensional projection of the sample, which is supported on a 
substrate plate made of a material transparent for incident radiation, the method 
comprising the steps of: 

(a) providing data representative of a two-dimensional projection of 
images of successive planes of the sample along the optical axis of an 
objective lens arrangement, which are continuously acquired by a 
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two-dimensional pixel array of a detector during a relative 
displacement between the focal plane of the objective lens 
arrangement and the sample along the optical axis of the objective lens 
arrangement through the sample; 
(b) processing said data by carrying out a two-dimensional deconvolution 
thereof with the predetermined Point Spread Function of an imaging 
system, thereby obtaining said image in the form of the two- 
dimensional projection of the three-dimensional sample. 
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